1. Field of the Invention
This invention relates to a system comprising multiple digital cameras viewing a large scene, such as a part of a city. Some of the cameras view different parts of that scene, and video feeds from the cameras are processed at a computer to generate a photo-realistic synthetic 3D model of the scene. This enables the scene to be viewed from any viewing angle, including angles that the original, real cameras do not occupy—i.e. as though viewed from a ‘virtual camera’ that can be positioned in any arbitrary position.
2. Description of the Prior Art
Existing ‘virtual camera’ solutions do not allow setting a virtual viewing angle (i.e. a viewing angle of a scene that does not coincide with the actual shooting angle of a real camera) without noticeable distortion. Nor do earlier systems keep the correct proportions of objects in the frame as the position of the virtual camera alters. Nor do earlier systems enable viewing angle amendments to be made in real time.
Also the existing solutions do not allow obtaining images from an arbitrary number of cameras that might be used in, for instance, an automatic multi-camera computer vision system. Automatic settings of each video camera in a multi-camera system—white balance, precise shutter opening speed synchronization, shutter, aperture and the frame rate—would be very different for all of these cameras, which makes working with and processing a set of images obtained from an array of these cameras very complicated.
Unlike the existing modeling systems (for example, lidar-based ones), the performance of implementations of this invention are based on conventional consumer-grade digital video cameras, which significantly reduces the cost of the solution. These implementations do not cause adverse health effects with emissions and do not interact with other active systems.                US 20140015832 describes a system reconstructing a 3D model from a set of video files obtained from shooting a single scene. The video files are synchronized with a global event (e.g. a flash) then the individual position of each camera is calculated for every frame. Presumably the video cameras are not fixed immovably. Then a 3D model reconstruction is processed, the textures are selected and mapped on the 3D model, and weather conditions are identified and simulated. The system is mostly intended for car tracking. Unlike the system we describe, this system does not rely upon permanently fixed cameras, the reconstruction is undertaken for each frame individually, which makes real-time processing in high-resolution impossible.        U.S. Pat. No. 6,940,538 describes a video synthesis system based on data obtained from a real time video camera and an additional synthetic object. The system contains one fixed video camera providing textures for a pre-defined model. A synthetic object is introduced in the model. The video is synthesized from the position of a real camera and the scope coinciding with the real camera and reflecting the synthetic object as a part of the scene. This solution relies upon one video camera usage, which does not allow shooting a wide scale scene or amending the viewing position from which the video is synthesized.        U.S. Pat. No. 6,674,461 describes a method of shooting a scene with a few video cameras; the footage obtained is mapped on a synthesized virtual scene. The approach described is designed for shooting a sole object, not an entire scene, so its authors do not provide a solution for fixing the lighting difference across the scene. The problem of setting up the cameras for real time shooting is not solved either. Also the system, unlike our one, has a limitation of 3 cameras only.        US 20120314089 is a method of time-based synchronizing a system of cameras. Auto calibration based on analysis of images obtained from the cameras is used here. Unlike our system, this method relies upon time-based synchronizing only; no solution for synchronizing white balance and exposure settings is provided.        EP1352521 describes a system of a few video cameras used for generating an optimal image of an object. Along with the cameras covering partial view only, a wide-angle video camera covering the entire view is used here. Unlike our system, this approach does not provide any solution for synchronizing of settings, and the wide-angle camera's settings are not used for controlling other cameras.        EP2695371 describes a system designed for 3D photo shooting, with light distributed evenly between a couple of cameras using a mirror, which achieves the same level of illumination for both of those cameras. Unlike our system, the method relies upon a mechanical synchronization (a mirror) only, no digital system of setting synchronization of two cameras is described (e.g., one camera's settings are not transferred to another one).        